Air conditioning control based on a human body activity amount

ABSTRACT

According to one embodiment, an image processing apparatus connected to a camera device that images a processing target includes: an image information acquisition unit; an accumulation subtraction image information creation unit; a feature amount information creation unit; and an action content identification unit. The image information acquisition unit sequentially acquires, from the camera device, image information formed by imaging the processing target thereby. Based on a temporal change of the image information acquired by the image information acquisition unit, the accumulation subtraction image information creation unit accumulates subtraction information for a predetermined period, which is made by motions of a person present in a room, and creates multivalued accumulation subtraction image information. The feature amount information creation unit creates feature amount information in the accumulation subtraction image information, which is created by the accumulation subtraction image information creation unit, from a region where there is a density gradient in the accumulation subtraction image information. The action content identification unit identifies an action content of the person present in the room from the feature amount information created by the feature amount information creation unit.

CROSS-REFERENCE TO RELATED ART

This application is based upon and claims the benefit of priority fromJapanese Patent Application No. 2010-038427, filed on Feb. 24, 2010; theentire contents of which are incorporated herein by reference.

FIELD

Embodiments described herein relate generally to an image processingapparatus, an image processing method, and an air conditioning controlapparatus.

BACKGROUND

In an interior space of a building, it is required to ensure anappropriate interior environment by air conditioning control with energyconsumption as small as possible. In the event of ensuring anappropriate interior thermal environment, it is important to consider athermal sensation such as heat and cold sensations felt by a person.

In the case where, in an amount of heat generated by the person (thatis, sum of radiant quantity by convection, heat radiation amount byradiating body, amount of heat of vaporization from the person, andamount of heat radiated and stored by respiration), a thermalequilibrium thereof is maintained, then it can be said that human bodyis in a thermally neutral state, and is in a comfortable state where theperson does not feel hot or cold with regard to the thermal sensation.On the contrary, in the case where the thermal equilibrium is disturbed,then human body feels hot or cold.

There is an air conditioning control system that achieves optimizationof the air conditioning control by using a predicted mean vote (PMV) asan index of the human thermal sensation, which is based on a thermalequilibrium expression. The air conditioning control system using thePMV receives, as variables affecting the thermal sensation, sixvariables, which are: an air temperature value; a relative humidityvalue; a mean radiant temperature value; an air speed value; an activity(internal heat generation amount of human body) value; and a clotheswearing state value. Then, the air conditioning control systemcalculates a PMV value.

Among the six variables to be inputted, those measurable with accuracyare the air temperature value, the relative humidity value, and the airspeed value. Since it is difficult to directly measure the activityvalue and such a clothing amount value, values set therefor are usuallyused. However, it is desired to also measure the activity value and theclothing amount value in real time with accuracy.

Accordingly, as a technology for measuring an activity amount of aperson who is present in a room, there is a human body activity amountcalculation apparatus described in Document 1 (JP 8-178390 A).

In the human body activity amount calculation apparatus described indocument 1, human body in a room is imaged by imaging means, a portionof human body is detected by detecting a shape (arch shape) of a vertexportion of human body from image information thus obtained, and anactivity amount of the person concerned, who is present in the room, iscalculated based on a moving speed and the like of such a portion ofhuman body. Therefore, the activity amount of the person can be obtainedwithout contacting the human body thereof, whereby accurate airconditioning control can be performed.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating a configuration of an airconditioning system using an air conditioning control apparatus of anembodiment.

FIG. 2 is a flowchart illustrating operations at a time of creatingaccumulation subtraction image information in the air conditioningcontrol apparatus of the embodiment.

FIGS. 3A and 3B are examples of an accumulation subtraction imagecreated by the air conditioning control apparatus of the embodiment.

FIG. 4 is an explanatory view illustrating a relationship between frameswhen the accumulation subtraction image information is created by theair conditioning control apparatus of the embodiment.

FIGS. 5A, 5B, and 5C are examples of accumulation subtraction images ofa person who moves at different speeds, the accumulation subtractionimages being created by the air conditioning control apparatus of theembodiment.

FIG. 6 is an explanatory view illustrating a state when feature amountinformation is created from a part of the accumulation subtraction imageinformation created by the air conditioning control apparatus of theembodiment.

FIGS. 7A and 7B are examples of image information acquired by the airconditioning control apparatus of the embodiment.

DETAILED DESCRIPTION

In general, according to one embodiment, an image processing apparatusconnected to a camera device that images a processing target includes animage information acquisition unit, an accumulation subtraction imageinformation creation unit, a feature amount information creation unit,and an action content identification unit. The image informationacquisition unit sequentially acquires, from the camera device, imageinformation formed by imaging the processing target thereby. Based on atemporal change of the image information acquired by the imageinformation acquisition unit, the accumulation subtraction imageinformation creation unit accumulates subtraction information for apredetermined period, which is made by motions of a person who ispresent in a room, and creates multivalued accumulation subtractionimage information. The feature amount information creation unit createsfeature amount information in the accumulation subtraction imageinformation, which is created by the accumulation subtraction imageinformation creation unit, from a region where there is a densitygradient in the accumulation subtraction image information concerned.The action content identification unit identifies an action content ofthe person, who is present in the room, from the feature amountinformation created by the feature amount information creation unit.

A description is made below of, as an embodiment, an air conditioningcontrol system that calculates an activity amount of a person, who ispresent in a room, without contacting the person by using imageinformation formed by imaging an interior (for example, an officeinside) as a control target, and performs accurate air conditioningcontrol by using the calculated activity amount.

<Configuration of Air Conditioning Control System Using Air ConditioningControl Apparatus of Embodiment>

With reference to FIG. 1, a description is made of a configuration of anair conditioning control system 1 using an air conditioning controlapparatus 30 of an embodiment.

The air conditioning control system 1 of the embodiment includes acamera device 10, an air conditioner 20, and the air conditioningcontrol apparatus 30. The camera device 10 is installed for eachinterior as an air conditioning target and images the interior servingas a target thereof. The air conditioner 20 performs air conditioningfor the interior concerned. The air conditioning control apparatus 30acquires image information formed by imaging the interior by the cameradevice 10 and controls operations of the air conditioner 20 based on theacquired image information.

For installing position and method of the camera device 10 for use inthe embodiment, a variety of modes are conceived. For example, thereare: a mode where the camera device 10 is installed, like a surveillancecamera, so as to image a space of the interior as the control targetfrom an upper end portion of the interior concerned at an angle oflooking down the space of the interior; and a mode where a fish-eye lensor a super-wide angle lens is attached to the camera device 10, and thecamera device 10 is installed so as to thereby image such an interiorspace from a center portion of a ceiling of the interior. Moreover, notonly a visible camera but also an infrared camera and the like areusable as the camera device 10.

The air conditioning control apparatus 30 includes: an identificationmodel information storage unit 31; an image information acquisition unit32; an accumulation subtraction image information creation unit 33; afeature amount information creation unit 34; an action contentidentification unit 35; an activity amount calculation unit 36; a PMVvalue calculation unit 37 as a comfort index value calculation unit; andan air conditioning control unit 38. Among such constituents of the airconditioning control apparatus 30, the identification model informationstorage unit 31, the image information acquisition unit 32, theaccumulation subtraction image information creation unit 33, the featureamount information creation unit 34, and the action contentidentification unit 35 function as constituent units of an imageprocessing apparatus.

The identification model information storage unit 31 prestores, asidentification models, a feature amount of the image information foreach of the action contents and a threshold value thereof. Thisidentification model may be created off-line in advance, or may belearned and created by on-line acquiring and analyzing informationextracted by the feature amount information creation unit 34.

The image information acquisition unit 32 sequentially acquires theimage information formed by imaging the processing target by the cameradevice 10 connected thereto.

The accumulation subtraction image information creation unit 33 extractssubtraction image information among a plurality of frames from imageinformation for a predetermined period, which is acquired in a timesseries by the image information acquisition unit 32, and createsmultivalued accumulation subtraction image information accumulated bysuperposing the extracted subtraction image information.

The feature amount information creation unit 34 defines, as a featureamount information creation target portion, a region where there is adensity gradient in the accumulation subtraction image informationcreated by the accumulation subtraction image information creation unit33, digitizes a feature of a brightness change of a peripheral region ofa pixel or block of the portion concerned, and specifies a positionalrelationship of the pixel or block of this portion on the imageconcerned. In such a way, the feature amount information creation unit34 creates feature amount information in the accumulation subtractionimage information concerned.

The action content identification unit 35 identifies an action contentof the person, who is present in the room, from the feature amountcreated by the feature amount information creation unit 34 by using theidentification model stored in the identification model informationstorage unit 31.

The activity amount calculation unit 36 integrates identificationresults of such action contents obtained from the accumulationsubtraction image information concerned in the action contentidentification unit 35, and calculates an activity amount of the personwho is present in the room.

The PMV value calculation unit 37 calculates a PMV value as a comfortindex value of the interior as the air conditioning control target fromthe activity amount of the person present in the room, which iscalculated by the activity amount calculation unit 36, and fromtemperature, humidity, air speed, radiant temperature of the interior asthe air conditioning target, and a clothing amount of the person presentin the room, which are acquired from an external sensor and the like.

The air conditioning control unit 38 decides a control value for the airconditioner 20, which performs the air conditioning for the interior asthe air conditioning target, based on the PMV value calculated by thePMV value calculation unit 37, and transmits the decided control valueto the air conditioner 20.

<Operations of Air Conditioning Control System Using Air ConditioningControl Apparatus of Embodiment>

Next, a description is made of operations of the air conditioningcontrol system 1 using the air conditioning control apparatus 30 of theembodiment.

In the embodiment, it is assumed that the feature amount of the imageinformation for each of the action contents and the threshold valuethereof are prestored as identification models in the identificationmodel information storage unit 31 of the air conditioning controlapparatus 30.

First, the time-series image information created by imaging the interioras the air conditioning target by the camera device 10 is acquired bythe image information acquisition unit 32 of the air conditioningcontrol apparatus. The image information acquired by the imageinformation acquisition unit 32 is sent out to the accumulationsubtraction image information creation unit 33. Then, the accumulationsubtraction image information is created by the accumulation subtractionimage information creation unit 33.

With reference to a flowchart of FIG. 2, a description is made ofprocessing when the accumulation subtraction image information iscreated in the accumulation subtraction image information creation unit33.

When the time-series image information is acquired from the imageinformation acquisition unit 32 (S1), filter processing for noiseremoval is performed according to needs (S2). For example, a Gaussianfilter is applied to this filter processing.

Next, the subtraction image information among the plurality of frames isacquired from the time-series image information for a predeterminedperiod while the filter processing is performed (S3). Binarizationprocessing is performed for the subtraction image information dependingon whether or not the acquired subtraction information exceeds thepreset threshold value (S4). Such difference-binarized image informationsubjected to the binarization processing is accumulated for each pluralpieces thereof, whereby the accumulation subtraction image informationis created (S5).

Examples of the cumulated subtraction image created as described aboveare illustrated in FIG. 3A and FIG. 3B.

FIG. 3A is an accumulation subtraction image created from thesubtraction-binarized image information created among the past imageinformation. Moreover, FIG. 3B is an accumulation subtraction imagecreated from subtraction-binarized image information created among thecurrent (up-to-date) image information and the past image information.As illustrated in FIG. 3A, in the accumulation subtraction image createdfrom the subtraction-binarized image information created among the pastimage information, a brightness distribution is formed so that imagelags can appear in front and rear of a shape portion of a person with ahigh brightness by a step-by-step density gradient. As illustrated inFIG. 3B, in the accumulation subtraction image created from thesubtraction-binarized image information created among the current(up-to-date) image information and the past image information, abrightness distribution is formed so that an image lag can appear in therear of a shape portion of the person with a high brightness by astep-by-step density gradient.

As an example of the above, with reference to FIG. 4, a description ismade of processing when the subtraction-binarized image information iscreated among the past image information, and the accumulationsubtraction image is created from the created binarized imageinformation. Here, it is assumed that frames 41 to 48 are acquired astime-series image information 40. Moreover, as parameters for creatingthe accumulation subtraction image information from a plurality oftime-series frames, a subtraction frame interval as an interval betweentwo frames to be compared with each other for acquiring thesubtraction-binarized image information is set at a three-frameinterval. A cumulative frame interval as an interval at which thesubtraction image information formed by the comparison at thisthree-frame interval is created is set at a one-frame interval. Thenumber of cumulative frames of the subtraction-binarized imageinformation for creating the accumulation subtraction image informationis set at three frames.

The parameters are set as described above, whereby, as illustrated inFIG. 4, subtraction information between the frame 41 and the frame 44 isacquired, and is subjected to the binarization processing, and asubtraction-binarized image 51 is created. Subtraction informationbetween the frame 42 and the frame 45 is acquired, and is subjected tothe binarization processing, and a subtraction-binarized image 52 iscreated. Subtraction information between the frame 43 and the frame 46is acquired, and is subjected to the binarization processing, and asubtraction-binarized image 53 is created. Subtraction informationbetween the frame 44 and the frame 47 is acquired, and is subjected tothe binarization processing, and a subtraction-binarized image 54 iscreated. Subtraction information between the frame 45 and the frame 48is acquired, and is subjected to the binarization processing, and asubtraction-binarized image 55 is created.

The binarization processing is performed as described above, whereby acolor subtraction, and the like among person's clothes, a background andthe like are absorbed, and a portion regarding the person's motion isstably extracted. Moreover, expansion or contraction processing may beadded in order to remove a hole and a chipped portion, which may becaused by the binarization processing.

Next, the created subtraction-binarized images 51 to 55 are accumulatedin a time axis direction by a predetermined number of cumulative frames,whereby multivalued accumulation subtraction images are created. In theembodiment, the number of cumulative frames is three frames. Therefore,as illustrated in FIG. 4, the subtraction-binarized images 51 to 53 areaccumulated, and an accumulation subtraction image 61 is created. Thesubtraction-binarized images 52 to 54 are accumulated, and anaccumulation subtraction image 62 is created. The subtraction-binarizedimages 53 to 55 are accumulated, and an accumulation subtraction image63 is created.

An image lag formed by a step-by-step density gradient of themultivalued accumulation subtraction images created as described aboveis narrowed as illustrated in FIG. 5A in the case where a moving speedof the person is slow, and is widened as illustrated from FIG. 5B toFIG. 5C as the moving speed of the person gets faster.

Accordingly, such parameters as the above-mentioned subtraction frameinterval, cumulative frame interval, and number of cumulative fames aremade variable in response to an environment and the action content,which are calculation targets of the activity amount, whereby the actioncontent can be detected with accuracy.

For example, in an interior such as an office inside where person'smotions are small, the subtraction frame interval and the cumulativeframe interval are increased, and in a space such as a department storewhere the person's motions are large, the subtraction frame interval andthe cumulative frame interval are reduced. In such a way, it becomeseasy to recognize an orbit of the movement of each person, and theaction content and moving speed of the person can be detected withaccuracy.

The accumulation subtraction image information may be createdsequentially in the time series, or may simultaneously create pluralpieces of the accumulation subtraction image information.

Next, a description is made below of processing when the feature amountindicating the moving speed of the person's portion is calculated in thefeature amount information creation unit 34 from the accumulationsubtraction image information created by the accumulation subtractionimage information creation unit 33.

As mentioned above, in the accumulation subtraction image created fromthe subtraction-binarized image information created among the past imageinformation, the image lags appear in front and rear of the shapeportion of the person with the high brightness by the step-by-stepdensity gradient. Moreover, in the accumulation subtraction imagecreated from the subtraction-binarized image information created amongthe current (up-to-date) image information and the past imageinformation, the image lag appears in the rear of the shape portion ofthe person with the high brightness by the step-by-step densitygradient.

Accordingly, in the feature amount information creation unit 34, in sucha portion of the image lag by the density gradient, which appears on theperiphery of the shape portion of the person, digitized brightnessdistribution information on the periphery of a certain pixel or block isdetected for a predetermined number of directional lines for eachpredetermined region, whereby a feature amount in the accumulationsubtraction image information is created. Information of this featureamount can contain: brightness values of the respective lines, which arewritten in order from the center to the periphery; relative valuesindicating brightness changes from adjacent pixels in the respectivelines; and data for coping with geometrical characteristics of thecamera device. The data for coping with the geometrical characteristicsof the camera device is positional information on the image, such as anx-coordinate and y-coordinate of the pixel concerned or the blockconcerned, and a distance thereof from the center of the image.Moreover, for the feature amount information, it is possible to performnormalization and weighting, which correspond to the variations anddistribution of the values, and according to needs, it is also possibleto perform enhancement of priority for brightness information effectivefor the identification, and to perform addition or summation forinformation regarding positions on the image.

An example of the predetermined region of the image lag portion, whichis defined as the feature amount creation target, is illustrated in FIG.6.

This region 70 as the feature amount creation target is a square regionwith a 15 by 15 matrix of pixels. Here, the case is illustrated, wherebrightness distribution information in lines of eight directions(respective directions of arrows 71 to 78) from a center pixel isdetected.

In the case where the feature amount is created for this region 70,brightness values are first acquired as the brightness distributioninformation sequentially from centers of the respective lines to theperipheries thereof, and further, relative values indicating brightnesschanges among pixels adjacent to one another are acquired from thecenters concerned toward the peripheries thereof.

Next, the sums of brightness values of the respective lines arecalculated from the brightness values contained in the brightnessdistribution information, and the brightness distribution information isarrayed in a clockwise or counterclockwise order from, as the head, theline in which the sum of the brightness values is the maximum.

In FIG. 6, it is determined that the sum of the brightness values of theline in the direction of the arrow 71 among the lines in the eightdirections is the maximum. Then, the brightness distribution informationfor the eight lines in the order of the arrows 72, 73, . . . , and 78 isarrayed like brightness distribution information 81, 82, . . . , and 88in the clockwise direction from the line of the arrow 71 as the head.

The matter that the sum of the brightness values of the brightnessdistribution information is large refers to a direction where thedirection approaches the shape portion of the person with the highbrightness. Specifically, it is estimated that the direction (directionof the line in which the sum of the brightness values is the maximum) ofthe arrow 71 arrayed at the head is a moving direction of the personconcerned. As described above, the moving direction of the person isestimated from the sum of the brightness values, whereby it becomespossible to identify the movement in every direction without holdingdependency on the moving direction.

Meanwhile, in the case where the dependency on the moving direction ofthe person is desired to be given, brightness values of the respectivelines may be extracted to be used as feature amount data withoutperforming such sorting processing based on the sums of the brightnessvalues among the above-mentioned processing.

As the digitized brightness distribution information on the periphery ofthe pixel or the block, there may be used brightness distributioninformation formed by combining information regarding peripheral regionsof a plurality of pixels or blocks, or information formed by combiningbrightness distribution information created based on accumulationsubtraction image information in a plurality of time ranges.

Next, in the action content identification unit 35, the identificationmodels stored in the identification model information storage unit 31are used, and the action content (standing, walking, running, and so on)of the person present in the room is identified from the feature amountcalculated by the feature amount calculation unit.

An example of the identification models is created by applying thesupport vector machine (SVM), the neutral network, the Bayes classifierand the like, which are typical methods for pattern recognition. The SVMis a method that is originated from the optimal separating hyperplanedevised by Vapnik, et al. in 1960's, and is expanded to a nonlinearidentification method combined with kernel learning methods in 1990's.For example, in the case of applying, to the SVM, vSVM to which aparameter v for controlling a tradeoff between complexity and loss ofthe model is introduced, v, γ, and the kernel are present as parameters,and these are appropriately selected, whereby highly accurateidentification can be realized.

Next, the identification results of the action contents obtained on thepixel basis or the block basis in the action content identification unit35 are integrated by the activity amount calculation unit 36, and theactivity amount of each person present in the room or a mean activityamount in the room is calculated.

Here, in the case where the activity amount of each person present inthe room is calculated, the activity amount may be calculated from theaction content identified based on the feature amount of the peripheralregion of the person's portion after the person's portion concerned isextracted from the image information. Moreover, processing such asclustering is performed for a distribution of the action contentsidentified based on feature amounts for person's motions in the whole ofthe room without performing the extraction of the person's portions, andthe person's positions are estimated, whereby activity amounts of therespective persons may be calculated. Moreover, in consideration that asize of such an imaging target on the image differs depending on apositional relationship between the camera device and the imaging targetconcerned, a frame for designating a region from which the activityamounts are to be calculated is set in advance on the image, whereby theactivity amount of each person present in the room may be calculatedbased on identification results in the frame.

Moreover, in the case where the mean activity amount in the room iscalculated, the mean activity amount may be calculated in such a mannerthat the activity amounts of the respective persons in the room, whichare calculated as mentioned above, are averaged, or that the meanactivity amount in the rooms is estimated from a relationship between anidentification result obtained from the whole of the image and animaging area without extracting the person's portions. In this case, theidentification result, the distribution and number of the persons in theroom, which are obtained from the image information, and the informationregarding a space of the imaging area are integrated, and an activityamount that is optimum for use in calculating the PMV value iscalculated.

Next, in the PMV value calculation unit 37, the PMV value as the comfortindex value in the room as the air conditioning control target iscalculated from the activity amount calculated by the activity amountcalculation unit 36, and from the temperature, humidity, air speed, andradiant temperature of the interior as the air conditioning target, andthe clothing amount of the person present in the room, which areacquired from the external sensor and the like.

Next, in the air conditioning control unit 38, the control value for theair conditioner 20, which performs the air conditioning for the interioras the air conditioning target, is decided based on the PMV valuecalculated by the PMV value calculation unit 37, and the decided controlvalue is transmitted to the air conditioner 20, whereby the operationsof the air conditioner 20 are controlled.

As described above, in accordance with the air conditioning system ofthe embodiment, the accumulation subtraction image information formed byextracting and accumulating the subtraction image information among theplurality of frames is created from the image information for thepredetermined period, which is formed by imaging the interior as the airconditioning target, and the image lag portion of the person's portionappearing on this accumulation subtraction image information isanalyzed, whereby the highly accurate activity amount is calculated.Then, efficient air conditioning can be executed based on circumstancesof the interior environment calculated based on the activity amountconcerned.

In the above-described embodiment, at the time of creating theaccumulation subtraction image information, there may be used not onlythe subtractions among the frames but also background subtractions,subtraction values in an optical flow (velocity field of object), movingorbit, affine invariant, projective invariant and the like of theobject, or physical amounts of these.

Moreover, it is also conceivable that the feature amount information isformed as follows.

In the image information of the imaged interior, in terms of thegeometrical characteristics of the camera device, as the imaging targetis moving away from the camera device, the size of the imaging target onthe image information becomes smaller, and as the imaging target isapproaching the camera device, the size of the imaging target on theimage information becomes larger. Therefore, a width of the image lagappearing on the accumulation subtraction image information is changednot only by the moving speed of the person but also by the positionalrelationship between the person and the camera device.

Hence, information on the geometrical characteristics (position on ascreen) of the camera is given to the feature amount information inorder to correctly detect the action content at any position within anangle of view without depending on the position of the person on thescreen.

Specifically, when the camera device is installed at the angle oflooking down the interior space from the upper end portion of theinterior, then as illustrated in FIG. 7A, the imaging target isdisplayed on an upper portion of the display screen as moving away fromthe camera device, and is displayed on a lower portion of the displayscreen as approaching the camera device.

In the case where the imaging target is imaged by the camera deviceinstalled as described above, the y-coordinate of the person is adoptedas the data for coping with the geometrical characteristics of thecamera device.

Moreover, when the camera device attached with the fish-eye lens or thesuper-wide angle lens is installed so as to image the interior spacefrom the center portion of the ceiling of the interior, then asillustrated in FIG. 7B, the imaging target is displayed more largely asbeing close to the center of the screen, and is displayed smaller asgoing toward the periphery.

In the case where the imaging target is imaged by the camera deviceinstalled as described above, a distance thereof from the center of thescreen is adopted as the data for coping with the geometricalcharacteristics of the camera device.

Then, the identification model to be stored in the identification modelinformation storage unit 31 is created by using the feature amounthaving the geometrical characteristics of the camera device, whereby theaction content identification is performed.

In the case where the camera device is installed on the upper endportion of the interior, and the imaging target is imaged at the angleof looking down the interior space, then for example, in order to copewith a motion in every traveling direction, the feature amounts arelearned in moving scenes including three types (distant, center, near)in the lateral direction and one type (center) in the longitudinaldirection, and the identification models are created.

Moreover, in the case where the camera device attached with the fish-eyelens or the super-wide angle lens is installed on the center portion ofthe ceiling, and the person is imaged immediately from the above, thefeature amounts are learned in moving scenes in the vicinity immediatelyunder the camera device and positions moving away from the cameradevice, and the identification models are created.

As described above, the action content is identified by using thefeature amount information having the brightness information and theinformation on the geometrical characteristics of the camera, wherebythe action content can be correctly detected at any position within theangle of view without depending on the position of the person.

Moreover, with regard to the learning of the feature amounts in thecreation of the identification models, in consideration of thegeometrical characteristics of the camera for example, based on afeature amount acquired based on an action scene in an environment wherethe size of the imaging target is large, it is possible to estimatefeature amounts in other circumstances. Moreover, based on an actionscene in the environment where the size of the imaging target is large,a video of an action scene in an environment where the size of theimaging target is small is created, whereby it is possible to learn thefeature amounts based on the video concerned. By using the informationthus estimated, types of videos to be imaged for the learning can bereduced, and the number of steps required for the learning can bereduced.

Here, the pixels or the blocks, which are taken as extraction targets ofthe feature amounts at the time of the learning, can be selectedappropriately, and it is not necessary that all of the pixels or all ofthe blocks in the image be taken as such targets. Moreover, it is notnecessary that all of the frames be taken as learning objects, either,and frames to be taken as the learning objects may be selected at acertain interval in response to the moving speed.

As described above, the identification models are updated by thelearning, and the feature amounts corresponding to the distances of theperson are learned, whereby expansion of a surveillance range in theimage information formed by imaging the imaging target can be achieved.

Moreover, it is also conceivable to set the feature amount informationas follows.

As mentioned above, the width of the image lag appearing on theaccumulation subtraction image information is changed not only by themoving speed of the person but also by the positional relationshipbetween the person and the camera device.

Accordingly, normalization processing is performed for the featureamount information in consideration of the information on thegeometrical characteristics (position on the screen) of the camera.

Specifically, when the camera device is installed at the angle oflooking down the interior space from the upper end portion of theinterior, then the y-coordinate of the person is used as the data forcoping with the geometrical characteristics of the camera device,whereby the normalization processing for the feature amount informationis performed.

Moreover, when the camera device attached with the fish-eye lens or thesuper-wide angle lens is installed so as to image the interior spacefrom the center portion of the ceiling of the interior, then thedistance of the imaging target from the center of the screen is used asthe data for coping with the geometrical characteristics of the cameradevice, whereby the normalization processing for the feature amountinformation is performed.

Then, the identification model to be stored in the identification modelinformation storage unit 31 is created by using the feature amountsubjected to the normalization processing by using the geometricalcharacteristics of the camera device, whereby the action contentidentification is performed.

In such a way, the action content is identified by using the featureamount information subjected to the normalization processing using theinformation on the geometrical characteristics of the camera, wherebythe action content can be correctly detected at any position within theangle of view without depending on the position of the person. Moreover,also in the learning of the identification models, the feature amountscan be estimated by performing the normalization processing. Therefore,the number of steps required for the learning can be reduced.

In the foregoing embodiment, the description has been made of the caseof using the identification models at the time of identifying the actioncontents. However, the mode for carrying out the invention is notlimited to this, and for example, the action contents may be identifiedbased on threshold values of the feature amounts for identifying therespective preset action contents (standing, walking, running, and soon).

While certain embodiments have been described, these embodiments havebeen presented by way of example only, and are not intended to limit thescope of the inventions. Indeed, the novel methods and systems describedherein may be embodied in a variety of other forms; furthermore, variousomissions, substitutions and changes in the form of the methods andsystems described herein may be made without departing from the spiritof the inventions. The accompanying claims and their equivalents areintended to cover such forms or modifications as would fall within thescope and spirit of the inventions.

What is claimed is:
 1. An image processing apparatus for processingframes captured by a camera device, comprising: an image informationacquisition unit that acquires image information including a pluralityof frames sequentially captured by the camera device in time series; anaccumulation subtraction image information creation unit thataccumulates binarized subtraction image information including aplurality of binarized subtraction images each acquired by subtracting apredetermined frame from a corresponding frame in the image information,the binarized subtraction image information being made by motions of oneor more persons present in a room, and creates multivalued accumulationsubtraction image information including a plurality of multivaluedaccumulation subtraction images each acquired by adding a predeterminednumber of corresponding binarized subtraction images in the binarizedsubtraction image information; a feature amount information creationunit that creates feature amount information from the multivaluedaccumulation subtraction image information, the feature amountinformation including a plurality of feature amounts, one feature amountcreated from a brightness change region where there are brightnessgradients in a corresponding multivalued accumulation subtraction image;and an action content identification unit that identifies an actioncontent of the persons from the feature amount information.
 2. The imageprocessing apparatus according to claim 1, wherein each of the featureamounts is created from digitized information formed by digitizingbrightness changes in a peripheral region of a pixel or a block in thebrightness change region in the corresponding multivalued accumulationsubtraction image, and from positional information of the pixel or theblock.
 3. The image processing apparatus according to claim 2, whereinthe digitized information created by a plurality of brightnessdistribution arrays each digitizing brightness changes of apredetermined number of pixels or blocks in each of a predeterminednumber of directions around the pixel or the block in the peripheralregion of the pixel or the block.
 4. The image processing apparatusaccording to claim 3, wherein the digitized information is created byarraying the brightness distribution arrays in clockwise or counterclockwise from, as a head, one of the brightness distribution arrays inwhich a sum of brightness values is maximum.
 5. The image processingapparatus according to claim 3, wherein the digitized information isformed by combining information regarding peripheral regions of aplurality of pixels or the blocks in the corresponding multivaluedaccumulation subtraction image, or by combining information createdbased on a plurality of corresponding multivalued accumulationsubtraction images in the multivalued accumulation subtraction imageinformation.
 6. The image processing apparatus according to claim 4,wherein the digitized information is formed by combining informationregarding peripheral regions of a plurality of pixels or the blocks inthe corresponding multivalued accumulation subtraction image, or bycombining information created based on a plurality of correspondingmultivalued accumulation subtraction images in the multivaluedaccumulation subtraction image information.
 7. The image processingapparatus according to claim 1, further comprising: an identificationmodel information storage unit that prestores, as an identificationmodel, identification information for each action content, wherein, byusing the identification model, the action content identification unitidentifies the action content of the persons from the feature amountinformation.
 8. The image processing apparatus according to claim 7,wherein the identification model contains feature amount information inanother imaging environment, the feature amount information beingestimated based on feature amount information acquired from an action ofan imaging target imaged in a predetermined imaging environment.
 9. Theimage processing apparatus according to claim 1, wherein based on athreshold value preset for each action content, the action contentidentification unit identifies the action content of the persons fromthe feature amount information.
 10. The image processing apparatusaccording to claim 1, wherein the binarized accumulation subtractionimage information is created by using any of subtractions among frames,background subtractions, subtraction values in an optical flow,subtraction values in moving orbit, subtraction values of affineinvariant, and subtraction values of projective invariant of an object.11. An image processing method using an image processing apparatus forprocessing frames captured by a camera device, the image processingmethod comprising: acquiring image information including a plurality offrames sequentially captured by the camera device in time series; basedon a temporal change of the image information, accumulating binarizedsubtraction image information including a plurality of binarizedsubtraction images each acquired by subtracting a predetermined framefrom a corresponding frame in the image information, the binarizedsubtraction image information being made by motions of one or morepersons present in a room, and creating multivalued accumulationsubtraction image information including a plurality of multivaluedaccumulation subtraction images each acquired by adding a predeterminednumber of corresponding binarized subtraction images in the binarizedsubtraction image information; creating feature amount information fromthe multivalued accumulation subtraction image information, the featureamount information including a plurality of feature amounts, one featureamount created from a brightness change region where there arebrightness gradients in a corresponding multivalued accumulationsubtraction image; and identifying an action content of the persons fromthe feature amount information.
 12. An air conditioning controlapparatus using a camera device installed in an interior as an airconditioning control target for controlling an air conditioner thatperforms air conditioning for the interior as the air conditioningcontrol target, the air conditioning control apparatus comprising: animage information acquisition unit that acquires image informationincluding a plurality of frames sequentially captured by the cameradevice in time series; an accumulation subtraction image informationcreation unit that accumulates binarized subtraction image informationincluding a plurality of binarized subtraction images each acquired bysubtracting a predetermined frame from a corresponding frame in theimage information, the binarized subtraction image information beingmade by motions of one or more persons present in the room, and createsmultivalued accumulation subtraction image information including aplurality of multivalued accumulation subtraction images each acquiredby adding a predetermined number of corresponding binarized subtractionimages in the binarized subtraction image information; a feature amountinformation creation unit that creates feature amount information fromthe multivalued accumulation subtraction image information, the featureamount information including a plurality of feature amounts, one featureamount created from a brightness change region where there arebrightness gradients in a corresponding multivalued accumulationsubtraction image; an action content identification unit that identifiesan action content of the persons from the feature amount information; anactivity amount calculation unit that calculates an activity amount ofthe persons from the action content; a current comfort index valuecalculation unit that calculates a current comfort index value of thepersons based on the activity amount; a control parameter calculationunit that calculates a control parameter regarding operations of the airconditioner from the current comfort index value; and an air conditionercontrol unit that controls the operations of the air conditioner basedon the control parameter.